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ABSTRACT 


Qptiiwm  non-linear  flltera  btilonglng  to  Zadah'n  class  - 

are  ;on«ldered»  Attention  is  restrlctsd  to  those  syutems  •whose 
present  output  Is  influenced  only  hy  a  portion  of  the  past  in¬ 
put-  The  input  signal  consists  of  a  msBsege  and  noise  both  of 
which  are  a-tatlonary  random  processes.  .For  this  class  of  f  JJ.'ters . 

it-  is  found  -that  only  -the  second  order  probability  density  func- 

/  • 

tions  of  ■the  sjessage  and  the  noise  are  necessary  for  obtaiiiing 
the  optimising  integral  equation.  It  is  assumed  -that  the  ampli- 

A 

tude  of  'the  input  tine  series  is  bouxided  and  takes  on  discrete 
•values  at  all  times.  This  assumption  is  not  -too  restrictl've  in 
practice  since  data  supplied  by  ccsaputers  and  devices  using  dlgi- 
■ted.  read-out  are  quantized.  3y  sv.bjec-c5ng  the  Joint  probability 
density  functions  ■to  a  few  mild  restrictions^  tt  is  fOnnd  -that 
■the  optimising  Integi-al  equation  I'educes  -fco  a  sys'ten  of  Integral 
equations  of  the  Wiener-Hopf  type.  By  viitiue  of  the  assuu^Jtlons 
.’Bade,  tbs  Fourier  transforms  of  -tbe  ke-mels  of  these  equations 
are  rational  functions.  A  me-thod  Is  developed  for  the  solution  of 
this  set  of  slmul-taneous  Integral  equations  and  three  exanqjles  axe 
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I.  OPTIMOM  lK)H-LlBgAR  TOCTaS 


1.  Introduction 

Broadly  opealcing,  optimaa  filters  and  predictors  are  devices  de¬ 
signed  to  produce^  upon  acting  on  the  pest  of  a  time  series,  a  va^'ue 
which  estimates  a  desired  value  of  8cb»  function  of  time  in  an  opti- 
mxn  fashion  according  to  scKie  fixed  eiTor  'irl.terlon.  The  most  con¬ 
ventional  optimum  criterion  has  heen  ihe  least  square  error  criterion. 
Adoption  of  this  criterion  has  often  re.f/jlted  In  equations  that  can 
be  haodlpd  analytically.  Wb.ea  the  injT.'.fc  time  series  is  siationaxy, 
the  classic  work  of  Wiener^  has  show/,  that  the  iiqnilse  response  of 
the  optimum  stationary  linear  filter  wMch  acts  on  the  infinite-  past 
of  the  input  is  the  solution  of  t!ie  WLener-Hopcf  equation.  Subsequent 
to  Wiener ' s  publication,  the  subj^scfi  t  f  optimum  linear  llltsrlng  and 

prediction  has  been  extended  in  many  iirections.  Aaong  them  are  the 

2  3  4 

’./ork  of  Zadeh  and  Bagazzixd,  Boctor,,  and  Davis  for  continuous  sys- 
terns  and  the  work  of  Blum,^  Chang,  end  others  for  saapled-data  sys¬ 
tems.  By  contrast,  due  to  the  lack  of  knowledge  of  characterizing 
a  non-linear  system  as  veil  as  the  inherent  difficulty  in  any  analyti¬ 
cal.  treatment,  relatively  little  woi'k  has  been  done  with  non-linear 

filters.  Oonditlons  under  which  op",Aaara  filters  for  the  detection 

n 

axid  prediction  of  signals  are  non-l  ..neax'  were  studied  by  Lanlng’  and 

8 

Ureaick”  onlj*-  for  discrete  data  po\.n'';8.  They  have  considered  the 

case  in  which  the  message  is  a  non-rsndan  linear  combination  of  known 

time  functions  i/lth  luiknovn  coefflclmts;  in  particular,  polynanlal 

g 

functions  were  considered.  On  the  ohM/r  hand  Zodeb^  has  outlined 
an  approach  which  is  based  on  the  cone Vderation  of  a  certain  system 
of  classes  of  non-Unear  filters.  He  bv.i,  derived  a  sequence  of  linear 
integral  equations  for  a  class  of  optlmivi  filters  and  has  shown  that 


-1 


-2 


as  the  filter  structure  becomes  zaore  coitipllcated,  nore  aod  isore  in- 
fonastlon  is  necessary  on  the  statistics  of  the  input  tine  series. 

A  rather  generalized  result  In  optlnmn  prediction  and  filtering  using 
the  least  square  error  criterion  was  derived  by  Rigachev/^^  a!he 
condition  which  he  derived  includes  all  of  the  optimizing  equations 
as  special  cases. 

Inasmuch  as  a  linear  filter  is  a  degenerate  case  of  a  non-linear 
filter,  Improved  results  can  usually  be  obtained  by  using  a  non-llncai- 
filter.  However,  we  often  find  in  practice  that  the  amount  of 
statistical,  data  necessary  for  the  design  of  non-linear  filters  far 
exceeds  \*at  is  available.  In  add3.t3.on,  the  ccotplexity  of  ths  struc¬ 
ture  of  the  filters  lea&a  to  problaiu!  that  are  unmanageable  by  ana¬ 
lytical  means.  In  order  to  circumvent  this  difficulty,  it  is  cosmton 
practice  to  make  slsqtllfying  assusqptions  about  the  characteristics  of 
the  message  and  noise  processes  as  well  as  to  restrict  attention  to 
a  certain  class  of  filters. 

'Ibe  optiamaa  filters  to  be  con8ider«»i  in  this  report  lie  within 
a  class  of  .filters  whose  input-output  relationship  can  be  expressed 

aa 

t 

y(t)  «  ^  Klx(t  -  T),T)dT  0<t<T  (l.l) 

Systmns  characterized  by  (l.l)  have  been  designated  by  Zadeh^  as 
constituting  the  class  !n.2>  class  is  a  m»id>er  of  a  sequence 

of  classes  of  non-linear  filters  designated  as  tTig^  o**  classes. 

Tt*  class  of  linear  filters  is  a  subclass  of  Consequent!;}  , 

every  class  In  the  hierarchy  includes  the  class  of  linear  filters. 

Ito  cite  an  example  In  this  class,  consider 


KtxCt  -  t),tl  ■  flx{t  -  T)]h(T) 


(1.2) 


..X 


’Aew'  f(a)  is  any  fuciction  of  its  ax-giJHsnt  and  hC"?)  iB  the  unit 
Jjrgemlse  .responEC  of  a  phyeica3.1y  rirealisattle  linear  filter.  f!b*n 


y(t)  «  I  ftxCt-'r)]h(T)a'S  0<t<T  (1.3)  . 

o 

.■spj.'j-'sents  the  output  of  a  systea  laraic.b.  consists  of  a  caEcafite'  arrange' 
i;!eat  of  an  ai'hia’ary  awro  r^wanory  ■.noa-liaea.T  device  follmwid  ’by  a 
filtai'  of  JBesisory  size  T. 

As  date  aujplied  by  cosaputers  and  fiisvl-ces  using  digit.!.''.]  .resd'-out 
qwsntlzKdj  it  imsm  logi-cai  to  J.nc;or3?ornte  thla  infoma'f.i.on  in 
tte  i,t3sign  of  f-Jj.'ters..  ‘Ihe  follo’irt.ng  s^isslysis  shews  'that,  a 

'•sc'iorl  Infomrition  can  be  ii£icd  finiltfnlly  when  the  form  of  tl»*  fil'ter 
is  given  by-  (i,.!).. 

- "  ■ibrroule.tion  of  the  QptlJiium  Precllc'i;or 

let  it  b?  tfCtpp<Jsi€.'i  that  the  5nx«r:  3rgnal  xCt).  bs-  '.i":  !?ur.a  of 
;vo  indep'enflr.n-:;  stetionery  :'-:«adca.  preot' — -  .'3twjs.ly;  lb'' 

«D.c.  the  s;vii3iielrK.l  ooiae 

xCt)  s-  mCt'fi  4- iiCtJ 

Tee  probi.eyi  :!.?  to  find,  a  filter  belong '.ar-:  te  tlie  cSru-is  i'‘ich  ttei 

chs  ■iii’ffi.rf'f.C'O  betvrcen  tte  ■■ict>.i:fl  outpi  .t  irtia  tbe  filter  'Xd.  n 
C'utpvt  is  xsiniwlKcd  :ua  scce^  s.'rr-j  ’.  U;f,  gbaf’t  1-  •’>! 

3>?nt  the  df-'i.r'-d  output  ■tsi.o.r?  q  is  £.’\v  .flmct.lon  of  ■■.tf?  'guxi'-i'nt 
v-hfi-."'  ’‘a.i-’o''-''.  -oysa.-  io  ■zta”v)  for  sll  t  .‘uid  let  sit'  b'  the-'  srr'or 
brt’>  u.'i.  fb.o  ce.jired.  cutyA/t  ind  tl.e  act  ■..•'I  outyr.U 


e;  t'i 


qU*'  t 


] 


l.,b 


i-r  vliefcoT'  i.i  used  here  o. 
in  th.'  "Xie  of  aoise. 


j.i  'MOii 


>}i  C'-' 


k 


As  in  Wiener's  tbeory,  it  vlU  te  postulated  that  the  predictor  is 
optiaum  \dien  l)  the  ensemble  mean  of  e(t)  is  equal  to  zero  for 
all  t,  and  2)  the  ensenible  varj.ance  of  €(t)  is  a  minimum.  Let 
us  denote  the  enseitfjle  average  by  <  Tbe  filter  is  therefore 

optimum  idien 


-  0 

(1.6) 

and 

®  lainiTmnn 

(1.7) 

Piatt  (1.5),  we  have 

2 

(1.8)* 

T 

-  2  K[x(t-T),T]dT  X  q[a(tw)]^ 
+  <^^qCa(t>}a)]| 


Upon  expansion,  (1.8)  becccee 
_  oo  00  T  T 

X  i-(x^,Xg)dT  dTgdji^dXg 
"00  -00  0  0  ^ 


^  X  q{mjj)  X  p(Xj^,n^)dT^dm^dXj^ 

-00  -00  o 
00 

+  I  (1*9) 

«oo 


«  We  assume  here  that  eoou£^  time  has  elapsed  since  the  signal  staited; 
consequently,  the  upper  limit  of  the  Integral  Is  replaced,  by  T. 


5 


\4iei«  In  ^  have  circuuventiBd  tto  lengthy  notation  by  letting 

*  xit  - 

Xg  ®  xCt  -  tg) 

«■  ai(t  +  a) 

pCxj^,Xg)  »  p[xCi  -  Tj_),  xCt  -  tg);  ~  Tg] 

»  joint  jarohabiUty  density  function  of  x(t  -  and 
xCt  -  Tg) 

B  p[xCt  -  mCt  +  o);  +a-] 

ra  joint  probability  density  function  of  xCt  “  “Tj)  slid 
fflCt  +  a) 

p(»^)  ®  ptx(t  +  a)] 

w  first  order  probability  denfdty  function  of  mCt  4-  a) 

Our  alia  is  to  find  the  kernel  K[xCt  -  t},t]  which  minircizes  (1»9)'’ 

The  mlnlnization  of  is  accomplished  by  the  usual 

technique  of  variational  oalcu  The  vailatiou  &I  corresponding 
to  an  adiiisslble  variation  SSK  in  K  is 

_  00  00  T  T 

BI  5  5  5  y  SS(Xj^,T^  )&K(Xg,Tg)  X  p(Xj  ,Xg)dT^d'rgdx3dXg 

^-CO  -00  O  0  ' 

S"  2p  fl  I  ^ 

-00  o 


(1.10) 

P  0,  vs  obtain. 


-  Loo  J 

^  evaluating  and  setting  it  to  zero  at 

for  all  adtaLBSible 


\  f  K(x2,Tg)p(xj,Xg)dr2CXg 


-00  o 


0 


(P  T 


^  Kix^,x^}  X  pCx^,Xg)aTgfiXg 
o 


-  5 


icx) 


Xp{ai^}  ^^*"a 


J 


Ci.ul 


lbs  desired,  cptlmlzing  Integral  ei^tloa  for  the  kernel  K  is 
therefore  of  tjrr,e  fom 


f®  0®  0^- 

\  q(m^)  XpCfflj>\)<3a|^  “  \  ]  KC:^,Tg)  X  p(xj^,Xg)a-r2dXg 

~O0  '■00  o 

0  <  <  T  {1.1S> 

In  order  to  ehow  that  {lol2)  gives  a  true  lainlmm  rather  then  a  mere 
stationary  solution,  ve  proceed  es  fo:Oj(y«iB„  Let  us  considar  a 
filter  QCx,t)  vblch  is  different  froa  K(x,'sf).  Inspection  o.f  Cl»9) 
showB  that  the  oean  square  error  resulting  from  using  the  filter 
^1Cx,t)  can  "be  aiiangsd  to  read 

-03 

00  00  T 

*  'i  'ik  i  K(xj^,t^)q(n^)  X  pC»^,X3^)dT^dm^aXj^ 

-CO  -00  o 

+  \  \  ^  f  -  Kix^,x,J] 

-00  -CD  o  O  - 


X  pCxj^^J^ldTj^drgdx^ebCg 


Cl" 13! 


7 


i^ch  Is  greater  tbaa  or  equal  to  the  aean  square  error  resultlog 
frcan  using  the  filter  K(x,t),  osraely 


since  the  last  tern  in  Cl°13)  1b  non-negative..  Consequently,  ve 
conclude  that  the  stationary  solution  does.  In  fact,  give  a  nlnl- 
oun  mean  square  error. 


3"  Qaantlzed.  Inputs 

As  loentioned  in  Section  I.l,  tlie  emplitude  of  the  input  tjUos 
series  aay  he  assuaed  to  he  hounded  end  discrete  at  all  times  in 
view  of  the  nature  of  the  measuring  device.  This  is  illustrated  in 
Figure  1,  vfaere  the  observed  input  is  denoted  by  x»(t).  It  is 
noted  that  no  cu sung^tion  is  «ade  regarding  the  amplitudes  of  the 
continuous  processes  ia(t}  and  n(t).  Our  analysis  includes  the 
special  case  In  idileh  the  anpUtudes  of  both  the  message  and  the 
noise  proctosses  are  bounded  and  discrete  for  all  t. 


in(t) 


to  the 
filter 


Fig.  1  Block  DtLagran  ghowlng  the  .tetual  Irawts;  x*(t) 
is  a  ftiaatlaed,  Tjae-contlnuous  31g>al 


8 


If  VC  let  the  nsanbcr  of  the  aaplitudes  of  the  irprt  signal  be 
2H,  then  the  Joint  probability  density  functions  pCx^,  :^)  and 
can  be  expiressed  as 


+N  4‘-W 

I  Z 
1^-3  jae-H 

i,J  0 


Ci)8C:^|  - 


(l»15) 


and 


S 

i^O 


Tj4a)  X  6(x^  - 


(1«16) 


vhsre  prCbabillty  that  x^Ct  -  r^)  equals 

and  xS'C  fc  ”  t^}  equals  Cy  and  f2.Cji^;  T3_4a)diH3  deao-t®s 

the  probBblU.ty  that  equals  *^Bnd  m(i>Hi)  lies  between 

b  and  ti>h£h”  Substifcutliig  (1-15)  arid  (X.l6)  into  (l.ll)  and  equat¬ 
ing  the  cori-esporiding  coefficiento  of  the  delta  functions  associatetl 
with  ;  v«  he.TC 


.oo 


5  ■^i<W’'*' 


•03 


4H 

E 

J«-N 

JjiO 


T 


yior  nrMtiary  \  j)  and  the  almultancoas 

ae'tj  of  integrol  equations  rln.  (l.lT)  is  far  too  convex  for  any  ana- 
l^ftical  aolutiiin-  In  fact;  it  is  improbable  that  a  direct  solution 
can  alwsiyi)  be  found,.  In  order  to  obtain  s,  reascmable  solution,  the 
follcurti-tg  assunptiono  regaiding  the  probability  density  functions  are 
msfle: 


9 


Ac  Joint  probability  density  function  of  the  neasagc  and 
of  the  noiaa  are  ayaaetrlcal  with  respect  to  tbeir  arguments  as 
veil  as  ssaoaetriesl  with  respect  to  the  origin,  namely 


It  follows  that 


“  PfflC^a’V  " 


^*ich  is  given  by 


CI0I8) 


(1«19) 


pCXj^^Xg) 


also  has  the  same  property, 
given  by 


X  PqCsc^l  “  Cl«20) 


On  the  other  hand  pCb^^  idilch  is 


(lo2l) 


is  only  symostrical  about  the  origin^  In  terms  of  the  expi'essions 
in  (I0I6)  aiid  (lolT),  this  assieption  con  be  written  eyaboUcally 

ftS 


Let  us  defim 

GO 

•"00  ^ 


(lo22) 


(;i..23) 


10 


since  foUowB  trm  (lo23)  that 

z^Ct^-w)  •  (1.25) 

Bo  it»r  all  1  and  J,  the  quantities  ^(j  |)  - 

■*1  \“^2  nui«2>®3^  of  deeayln^^  ia:i;$oiientlal8°« 

It  Is  the  logical  extension  oT  the  usual  assusqatlon  that  the  auto¬ 
correlation  fixiction  vhleh  la  given  by 

is  the  em  of  exponential  functions. 

In  view  of  aasuBgitian  k,  the  optiDun  filter  can  be  repreetanted 
by  the  stroetuze  shcvn  In  Flguoas  2. 


yijc...  2  ScbaaKtlc  ItepreBeatat-ioa  of  a  ]fon-lln»ar  Kilter 


«  Hixla  Is  a  weaker  condition  than  that  vhich  rsquiies  a31  A„ 
to  be  fscaying  ejqponeatlala. 
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IQ  Figc  1, 

filtered 


are  tlia  unit  ispulee  reqpoaaeB  of  linear 
becoaea  linear  vhen 


KCc^.t) 


KCCj.,-!') 


for  all  J  and  ko 


ni  see  the  valldil^  of  Fig»  1,  it  is  necessary  to  shov  that 


(1.26) 


KCc^"?)  a  t) 

Fran  (lo6}  'ue  conclude  that 


6  ml,  2,  o..,  H  (1.27) 


<y(t)X^  ”  0 


(1.28) 


CooaequsxiUy 

<K(z»(t  «  'r),'r3^^  a  0  (1.29) 

Let  f,  «  the  prcibaibility  that  x»Ct  »  t)  tatass  on  7aa,ua  c  . 

J  J 

Then. 

f.  »  f  j  J  ooo^  H  (Xu^O) 

vhich  can  be  deduced  fTcu  our  p:ra7laus  aesmopitioDS  on  the  process 
xS'Ct).  Bq.  (1.27)  heccBec 

2fjkCc^,‘r} +kCc_,/0|  0  (I.?!) 

^  '  i 

In  order  for  (l.Jl)  to  hoM  for  my  set  of  satisfying  ^  f ^  *  1, 
it  is  necessary  that 

KCc^,-f)  +  X(c_j,t)  »  0  c’*l,2,  H  (1.52) 


12 


vhlch  vitli  Clo27)‘  Sq*  Cl«l8)  can  aow  be  vritten  as 

C  XKCcj,-^)dt2 

i  a -1,1, -•,11  (103) 

By  using  (l«22)  and  (l',25),  it  is  seen  that  H  of  lie  2H  equations 
In  Cl«33)  «2'e  redundant.  Let  us  replace  r,  and  by  t  and 
T  respectively,  and  let 

v^j(|t-t|)  «  A^^jCjt-T|)  -  A^__^_j(|t-Jj) 

K^ir)  a  K(e^,r  } 

Bq*  (lOJ)  finally  reduces  to 

2.(tHa)  »  E  f  V  {jt~if|)K,CT)dT  (1.36} 

a.  «o  ^ 

0<t<T;  lal,2,..c,H 

nroB  (1^9)  ve  see  that  the  ndniaian  wan  square  error  is  given  by 

B  T 

'2^1  8^(tw)l^(t)dt  (1.37) 

i"GL  ’’o 


(1.5^) 

(1.35) 


4« 

«  S 

^  ^  d— 5 

Jsto 
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4„  Method  of  Solution 

It  was  shown  in  the  previous  aeetioo  that  Is  the 

sum  of  a  number  of  deca^ring  e^qmentlal  functions;  hence  its 
^burier  transform,  defined  as 


C  VI' 


2»tf 


'’-00 


is  a  rational  function  of 


Suppose* 


(1»38) 


(1-39) 


-where  D. 


“ij 

is  of  order  n 


axe  constants,  is  of  order  d  and  P.  .ex'*) 


ij 


(n„<a). 


ir 


Ve  shall  der!.-ve  in  this  section  a 
necessary  and  sixfficient  condition  under  wMch  a  unique,  absolutely 
Integrablc  solution  of  Clo36)  exists.  Kj(t)  is  ahotJlufcely  integrable 


if 


for  J  ds  l,2,ttw5>,H 


(l..to) 


which  i.s  the  usual  stabi3J.ty  condition  for  linear  sjngtaosc.  A  system 
is,  for  our  purposes,  defined  to  be  stable  if  all  bounded  Jaiyuts 
imsult  in  bounded  outputs-  Oui’  result  will  also  indicate  that  a 
formal  solution  can  always  be  ebtaisKsd  if  tljo  restriction  Imposed  by 
(l.^)  is  rvHsoved.  The  a3)proac:h  here  is  first  to  transform  (I.56) 
to  a  s.'iJnpler  system  of  Integrad.  equations.  It  is  then  ;diown  that  the 
solution  of  the  modified,  sjstan  of  integ:ral  equations  does,  in  fact, 
satisfy  (1.36).  When  convenient,  the  fo-Hanib^  notation  will  be 
used: 


*  The  conmon  de.n.miinator  of  til 


baa  been  usad. 
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[U] 

- 

KCt) 

z(t-jor)  « 

zIJMSl  «* 

w(|t|)  *• 

T 

C  v(  t-r| 
o 

[DP(  %] 
et- 


tax  N  X  IS  square  oatrix  whose  elesaents  are  (this 

mtrlx  la  assumed  to  be  aon-siiigular) 
inverse  of  [d] 

a  column  matrix  idiose  elements  are  time  functions 

J  Es  1^2^  •  tt  o 

a  columtL  zaatri:^  elitanents  are  tlae  fuzictlans* 

ZjCWa),  j  a 

a  column  matrix  tdios'S  elasents  are  time  functions, 
y  Ct40t),  4  «  1,2, ....,11.. 

2*' 

inverse  Pburier  transfoim  of  1/qC\  ) 

?  A 

)K(T}dT]  B  a  column  matrix  whose  elensnts  are  \  w(|  t-Tj)K^(T)dT; 
i  “  1,2,  »  o  •  ,Ba 

tx  an  N  X  H  square  matrix  whose  elements  are  linear 
dS 

operators,  j  C”  ;  i  4  »  1, 2,  = . . ,  H. 


Lpv  yCt-So)  be  any  solution  satisfying  the  following  system 
of  dlff/erential  equations: 

zCbKK)  »  Is  ?C- yCtic)  0<t<T  Cl.4l) 
dt'^ 

aal  let  the  acdifled  set  of  intei^ral  equations  be  given  by 

T 

yCt-jq)  «=  Cd][^  ^-^It-TOKlTldT]  o<t<r  Ci*42) 

0 

Me  will  now  show  that  the  solution  of  (1.42}  also  Batisfies  (I.36) 

®o  see  this  we  pre-multipily  both  sldescf  (1.42)  by  which 

becoBies 

[  I  wfj  t-T5  MT)dT]  «  [D]"^  y(tHiCt)  0  <  t  <  T 
0 


(1.43) 
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Operating  on  both  sides  of  Cl*>*3)  [D  P  (-  yields 


[nP(-5^)][f  Odt-tD^d-r]  -  [DP(-^)][D]-^y(bw) 

dt^  *o 


dt*- 
0  <  t  <  T 


(l.W 


tOie  left  side  of  (1.44)  is  a  coIubei  matrix  vhose  eloserrts  are 


/.I 


1  ®  1>2>  • 9 c f  H 


vhlch,  as  shown  in  Appendix  k,  becomes 


E  T 

S  \  w.,C|t-Tj)K.(-r)dT;  i  - 
J«1  o  ^  ^ 

Equation  (1.44)  therefore  reduces  to 


^  S  V  .(tt-T|)K.(T)dT 

J«1  ”0  ^  ^ 

which  is,  in  fact,  (I.36). 


Zj^(4W);  0  <  t  <  T 

1  *  X^S^eev^iR 


(1.^5) 


In  section  1.2  we  have  shewn  that  any  solution  (if  it  is  not 
unique)  of  (1.36)  will  give  the  sene  mean  square  error.  It  Is, 
therefore,  losiaterial  whether  other  solutions  of  (I.36)  exist  which 
do  not  satisfy  (1.40).  Eswrtheless,  ae  we  will  show  in  Appendix  C, 
all  solutions  of  (1.36)  are  necesssrily  the  solutions  of  (l.40}. 

We  shall  now  investigate  the  solution  of  the  modified  systen  of 
integral  equations  (1.42).  It  is  shown  in  Appendix  A  that  if  a  solu- 
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tlon  exists^  it.  BEtisfles  a  set  of  slnultaoeous  differential 
equations 

QC“  y(taa)  «  rPlltft)  0  <  t  <  T 
dt^ 

The  solution  of  (1.46)  can  he  vxitten  as 

K(t)  ®  qC-  iltjc)  0  <  t  <  T  fl..,47) 

dt*^ 

Since  tbe  infoimtioa  jartainUag  to  the  derir'atives  of  at 

t  o  0  Rnd  t  »  T  is  not  inoluded.  in  tbe  derivation  of  (1.46)^  we 
find  that  certain  ecaidltions  on  .yCtSo)  are  necessary  in  order 
thsn  the  so  obtained  ftor.  (1.47)  do  satisfy  (1.42).  Those 

conditions  are  obtained  by  substituting  (1.46)  into  (1.42)  and  solv¬ 
ing  the  resultant  equations  as  en  identity.  !I5ie  developineat  parallele 

UL 

that  appeaxlog  in  Appendix  2  of  Oavenporc  and  Hoot. 

Let  us  first  establish  a  useful  result.  Fnan  the  dfifinltlon 
of  v(St\),  that  is. 


^1  ti) 


3^ 

2x 


[  d\ 

*-oo 


(1.43) 


ve  obtain  the  corre«pondi.ng  differential  equation 


Q{— 2)^(iti)  -  m 

fit*^ 


(1.4^?) 


In  poirticular,  let 


d 

z 

TsuO 


23t. 


1 


Cl«50] 


3.7 


Bq«  (1o49}  can  bS'  aB> 

r  c-if  (Jtfl)  «  Ht) 

fcsO 

This  i-elation  vill  be  used  later  on.. 

Oeing  (loJO),  Sq.  Cl.J»6)  can  ba  rewritten  as 


(1-51) 


[D] 


(-«’■  iLaal 


0  <  <■  <  T 


Moltiplyiag  both  aides  of  (1.52)  by  w(jt-T!)  and  Ijotegratlng 
the  resultant  exprasaion  frcm  0  to  T,  we  obtain 


tD]t  f  w(»tet|)K(T)dT]  ^  f  £  C-3-)\v  wCt“T)dT 

0  'o  }C«0 


C  <  t  <  T 

Eere  we  have  separated  the  rangis  of  integration  into  two  regions; 
for  0  <  t  <  t,  the  kernel  is  vCt  •>  while  for  t  <  “?  _<  T, 
the  kernel  is  w(r-  t).  After  integrating  the  right-hand  aide  of 
(l»53)  by  parts  2d  tijaes  and  laaklrg  <we  of  the  property  tlset** 


we  are  left  w3.tb  integrals 


M)’'  9a 


^  t-T|„idt 


♦  w^^'fjt'j)  denotes  the  2k'^  deri-vatlve  of  vOtIf. 
See  Appendix  A-, 
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unlntc^rated  as  they  occur  at  every  other  step.  In  addition,  terns 
involving  the  derivatives  of  y{t4a)  at  t»  0  and  t  ■  T  are 
carried  over  frcn  each  of  the  2d  integrations. 

It  Is  shorn  in  Appexidix  B  that  the  rl{^t-hand  side  of  (l.5$ ) 
can  he  expressed  as 

^  w^^"^^(t)  •  Y  + 

£«1  *  1*0. 

o  kfiO 

where  Y^  and  are  column  matrices  whose  elenents  are 

and  Z^  (i  «  1,2, Y^  and  Z^  are  linear  ccid>lnatlons 
of  thfj  derivatives  of  yjj^(t+a)  (i  «  1,2,..., B)  at  t  ■  0  and  t  »  T 
defined  hy  (3.8)  and  (B.9)  respectively.  By  (l.^l),  the  8U»- 
natlon  in  the  last  integral  of  the  above  eagpression  is  >  t). 

jOsnce,  using  the  property  of  the  delta  function, 

8(t«T)y(TW)  dT  ■  y(t<a)  (1.55) 

o 

Bq.  (iv??)  can  be  reduced  to 

[»][?  v(|t-tO  KCE)dT]  .  yii>ia)+  £ 

*0  l«a 

(la 

0  <  t  <  r  (1.36) 

He  observe  from  (l«36)  that  in  order  for  the  solution  obtained 
from  the  differential  equation  (l.^)  to  satisfy  the  systm  of 
integral  equations  (l.^),  means  hhoold  be  provided  to  taka  car*  of 
the  additional  taras 
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Let  us  add  to  the  solution  of  tbe  differential  equation  two  terns  of 
the  f  one 


bj^8(t)  +  Cj^sCt  -  C)» 

ubere  and  are  column  mtrices  consisting  of  eleaients 
^  for  j  a  1,2, Than  ib  can  be  easily  shown  that  (I.56) 
is  identically  satisfied  if 

Z  Y.  »:<  vCt)0 

and 

T  Zy  »  w<T-t)H 

M 

where  §  and  H  are  colixm  matrices  defined  by 


G  -  [Djbj  (1.59) 

and 

H  »  [Djcj^  (1.60} 

It  will  now  be  shown  f^at  a  necesaazy  and  sufficient  condi.tlon 
that  the  solution  of  (l<36)  be  u'd.que  and  tdisolutely  Integrable  is 
that  y{-Wa)  obtained  fran  (1.4i)  have  a'(2d  -  2)  undetenoined  con- 
staots.  This  means  that  tie  detezninant  |  JXF(K^)  |  is  a  poIynoBdal 
of  degree  5(2d  -  2). 


«  This  does  not  violate  Eq,<  since  {  j  6(t)j  dt  «»  1 

'-00 


(1.57) 

{I.5S) 
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Me  note  firm  CB.8)  aarl  (B=9)  tJiat  and  are  .linear 

functions  of  yCtKii}  and  Its  eiu:ce8s;J.ve  derivatives  evaluated  at 
t  «  0  and  t  as  T  respsctively.  Let  m  assiaaf*  that  y{t^}  has 
If  (2d  -  2)  wndetemined  coefficients »  Sben  both  and  can 

be  es^^'ssed  as  linear  combinations  of  the  sans  nui^r  of  coefflciei:tt@. 
Frcsn  (l.W),  vs  see  that  v(t}  can  be  eacpressed  as 

v(t)  «  r-  t  >  0  Cl»6X) 

i«l  ^ 

,Xt  folinws  that 


d 

r. 

issl 


#~i 


t  >  0  Ci.i,-''-' 


Upon  eqiiating  the  co.rreeponding  coefficients  of  (i  1,2, 

In  (I-ST)  a»d  {l»58),  w€-  obtain  2®.  elgebraic  'Sijuationao 
Since  tlia  nvsobsr  of  untaaowns  is  ulx-io  2W.  —  "■  which 

belong  te  sad  Z^  and  the  rt®f.vJitdj:ig  2N  isre  contilSmted  by 
0  and  H  ■*”  a  unique  solution  can  always  be  obta!,;.’.ed.. 

Sc>  far,  we  have  on3.y  considered  tlr-  finite  assnory  f  iltei”.  03ie 
result,  however,  can  be  appU-ed  directly  to  the  lafinlte  aieiaory  filtc:;-, 
in  whi.ch  T:  .is  infinite.  Jtor  thw  inf-Iaite  aaoaory  filter,  the  sta¬ 
bility  condition  beeoraes 

fiO 

\  }K,Ct)}cit  <;  a)  (1.6,'.) 

is  ^ 

Eq.  (1.63)  lii^les  that 

A.  ‘JChe  roots  of  the  equation 

!ap(A®)l  0 


cannot  Is  par£*ly  dmI,  and 
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B.  Tbe  taims  In  KCt)  -wkS.cli  belong  to  the  roots  of  {lo64) 
that  lie  in  tbs  lower  half  of  the-  \  plan®  should  be  disoardedo 

When  eondition  (A)  is  satiefied,  a  unique  solution  again  can 
be  obtalr!(@d°  In  the  infinite  nenory  case^  it  is  necessary  to  use 
only  (l«^7)«  Oils  nusiber  of  equations  as  veil  as  the  nurahar  of  uhlmowne 
are  rsdticed  by  a  fatJtor  of  2. 

As  a  final  rjjnark,  we  see  fi'am  our  result  that  it  is  not  neces¬ 
sary  for  yCt^)  to  hove  !lC2d  -  2}  undetexiniriad- constants  the 
ataljiJLitj'  conditions  —  (l.-’jO)  for  the  finite  xaenory  filter  and 
(1.63)  for  the  infinite  mmiy  filter  —  are  r<aaov®do  In  this  case, 
we  can  a3.ways  obtain  a  forssal  solution  by  adding  delta  fuxictions  and 
higher  older  deriTOtlvea  of  delta  functions  to  the  solution  obtaiiJed 
from  the  differential  equations.  The  successive  derivatives  of 
della  functions  are  defined  by 


f  8(0  (t  -  t^)fCt)dt  ..  (-1/  (t^)  (1.65) 

*00 


As  an  exenqile,  consider  the  solution  of  (I.56)  \dien  all  P., }  «  1. 
It  can  be  ahown  that  (1.56)  will  be  identically  satisfied  If  we 
add  to  the  solution  of  £Ct)  obtained  from  ■*  he  additional 

teme 


ipl 


4* 


and  choose  ^ 

and  C£  BO  that  the 

foUavflng  equations  ore 

satisfied; 

“ 

1,2,..., 2d 

(1.66,) 

and 

“'!e 

N| 

IS 

1,2,..., 2d 

(1.67) 

22 


Hie  physical  sigolficance  of  th«  bl/jbar  order  impulse  functlotia  is 
that  the  filters  are  required  to  perfozm  differentiation  operations. 

At  least  tvo  procedures  can  be  adopted  to  obtain  the  solution 
of  Ci<.36}.  For  the  numerical  examples  in  the  next  section,  the 
foUoving  procedures  have  been  adopted . 

1.  Use  Cl»4l)  to  obtain  the  hoaogeneous  and  the  particular 
solution  of  y{t-»a). 

2o  liom  Cl.l^7),  ve  find  KCt)- 

3-  Add  to  K(t)  two  terns  of  the  fona  b^  £(t)  +  c^6{t-!r). 

4.  Substitute  the  solutions  obtained  in  steps  (2)  and  (5)  above 
into  (1.36)  and  perform  the  integration..  1!he  unspecified  constants 
are  determined  by  solving  the  resultant  set  of  algebraic  equations . 

It  should  be  noted  that  Kft)  obtained  from  steps  (l)  and  (2) 
above  is  esasntlally  the  solution  of  the  set  cf  dlfferentiad 
equations 

-  Q(-  zOm)  (1.68) 


3.  Enaaples 

1.  In  this  exan^le  ve  consider  a  prediction  problan  in  the 
presence  of  noise.  Let  m(tio)  'be  the  desired  output.  lOie  message 
m(t)  Is  a  zmndom  square  wave  taking  on  the  values  il.  with  equal 
ptiobabili  by.  !!h«  probability  density  of  tbs  time  duration  between 
transitions,  t,  is  (a  Bolsson  process).  !Die  second  order 

proibablllty  density  function  of  the  taesaage  is  given  by 

PmK'  i)[(l,-l)  +  (-1,1)] 


(1.69) 


where  Ci,u)  d^aotes  6(iii^  -  -  j)o  In  other  vorfle  It  Is 

the  point  where  Jn(t  -  takes  on  'value  1  and  in(t  -  Tg)  takes 
on  value  j.  The  probability  density  of  the  noise,  n(t)^  is  of  the 
fame  form  wl'fch  0,  replaced  by  3^,.  W®  aheill  choose  3^^  *  2, 


pg  «  4,  and 


3^  replaced  by  3-. 

-2Q  1 

■  'I'o 

2 


.'E^  ufling  (1.20)  and  (1.21),  we  first  ob-taln  pCx^,Xg)  aisd 
p(n^,x^)  widch  upon  substituting  into  (1.12)  yields  two  Blmul-fcaasoue 
ixi-tegral.  e(,-',uationa.  Inspection  of  'the  '.resul'fcant  tolegral  equations 
a'oowB  that  K^Ct)  s>  0.  Die  integral  eqmtion  for  KgC't)  beccoes 


-2t 


I 


[e 


-2I4"T|  g-4lt-T! 


r)dT 


The  Pourler  transform  of  the  kernel  is 


0  <  t  <  T  (l/i-'O) 


„(x=,  .  mp.  .  . 

qCx*^)  \  H!-  c>C^^  +  64 
Prow  step  1  of  the  procedure,  we  have 
2 

C"  ^-o  +  8|yCt)  “  0  <  t  <  T 

dn 


Taerefore, 


.y(t) 


-2t 


0  <  t  <  T 


Cl.^Ti.) 


(1.73) 


St<sp  2  yields 

i^Ct]  -  -eg;  [Oj^j^e"^^’  t  0<t<T  (17^^ 

We  now  add  to  K^(t)  the  'tenns  bj^S(t)  +  and  substitute 

the  cc*i5)lete  «xpr®sDion  for  KgCt)  into  (1.73).  Asaanlng  T  » 

100  Billiseconds,  we  find  that  the  integral  equation  can  be  satisfies’ 
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vben  the  uolciienms  «re 

«  -185  X  10"^  j  «  420  X  10“^ 

(^  .  -3  X  10'5  ;  »  63  X  lO"'^  (1„75) 


Hexice 

K^Ct)  *  0 

K^Ct)  a  0«426e‘'^^  OoOOSe'^^  +  0„42B(t)  +  Oo0636(t-0.l) 

0  <  t  <  0.1  (1.76) 


KgCt)  «  0  f or  t  >  Ool 


S.tnce  K^Ct)  ^  0,  the  <q?t:lauBi  filter  takes  the  fona  of  a  linear 
filter  whose  unit  Inpulse  responae  is  g  K^Ct)  for  0  <  t  <  Od  and 
Is  zero  for  t  >  O.l.  IMa  is  in  fact  the  optlBun  linear  filter  since 
(1.70)  is  actually  the  Modified  Wlener-Hopf  equation  T*ere  the  iqpiper 
liJKlt  on  the  integral  has  been  changed  from  00  to  T. 

2<’  Here  ve  consider  a  pure  prediction  problem,  namelj'  x(t)  •> 
aCt).  Tte  desired  output  is  again  ■(tHo).  We  assuae  that  the  second 
order  probsbility  density  of  the  message  is 

^[1  -  e-Pl‘^2-nL  e-P2W2-ril  „  e'^Pl+^^'^2’''l‘][(2,-2)+(l,-l)+<-2,2>{(-l,l)] 

+  ^[1  +  e"^'^2"ril.  e-CPl-*^)' T2-»iI][(2,-i)+(i,-2}4/..2,iM-1,2)] 

+  .  e-Pl»‘^2-'riJ+,-P2tT2-Til.  e-CPl-^)»T2-Tilj^g^^j  +<l,2)+U2rlW-V2)l 

+4g[l  +  «’Pll‘»2’nl+  e-P2i^2-riJ.+  ^2’'"3I][(2,2W,1,1;H<-2,-2>K  -1,-1)] 


(1.77' 
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Tbie  situation  arises,  for  instance,  when  the  message  is  the  svm  of 
tvo  Independent  Poisson  processes  vbose  anrplitudes  take  on  the  values 
•t  ^  and  "t  ^  respectively.  The  second  order  probaMlity  density 
function  of  a  Poisson  process  is  of  the  form  given  by  (l.6S;'')» 

In  this  example,  let  us  allow  the  filter  to  have  an  infinite 
memory.  To  simplify  the  calculation,  we  choose  «  2,  pg  «  1. 

The  prediction  time  a  is  arbitrary.  Let  us  denote  e“®  by  k  (O  <  k 
<  l).  Upon  substituting  this  Information  into  (l.l2)  we  obtain  the 
following  two  simultaneous  Integral  equations. 


+  3ke"^  »  \  e"'^"'^Kj^(T)dT 

o 

o 

+kV^  +  3ke"'*^  e 

o 

+  \  e"l^”'^*]ligCT)dT  t>0  (1.70) 


It  is  easily  verified  that  the  solution  is 
K^(t)  =.  I  C3k  -  k®)5CT} 

KgCx)  »  I  (5k  +  k^>o(T)  (1.7!?) 


It  is  observed  that  the  cajly  solution  for  which  K^(t)  «  2I^(t) 

[a  linear  filter)  la  when  k  >  1,  which  correapooda  to  the  trivial 
case  of  zero  prediction  time.  Jtr  any  finite  prediction  time  a,  a 
oon-llnectr  zero  memory  filter  reeulta. 


By  using  {X«37),  th«  nonaoliz'^d  sqmire  ex. nr  can  be  ewslu- 
ated  as 

k'^Avl  *  *  ••  2I  f'-g  ^  i>“'‘ii£i(t)dt 


'r]\ 


00 


I  [  I  4  I  .te“''^]Kj,Ct)dt j 


|o.,9k^  4-  j 


{'.i  oeo'i 


i'OT  -iihe  aaks  of  ccugMS’lsoa,  tie  mean  square  error  of  the  linaai' 
filter  is  also  obteinedo  5!b<!  eutocorralation  f>iac.t.lon  of  the  laes- 
sage  is 


yhu  - 


The  Wlener~Hopf  equation  becemes 

.00 


56ke'’^ -r  =<6  ^  bh(T)dT  t>0  C;i.,88) 


T.h«  solutJoii  of  this  equation  ie  fcimd  to  be 

h(T)  ®  0,12Ck  -  +  (0.13S5k^  +  0.86lk;i5(t) 


'■'loS'Jj 


Tiae  corresprjnding  nonnallzeci  mean  E.qaare  eixor  is  gL'fen  by 

k'’>Av]  -  1  -  53  r ‘f' ‘  ‘f  +  IS 


JL-^.near 


2i  “•  p 

^  1  -  \0.012k  4  0,176k-'  J.  O^eiak*^] 


.'nsx-.ictioji  of  (1.3-H'  and  (l.GO)  shown  Ijhak  fl.Sk} 
greater  thiU  or  rqual  to  (1.30);  that  Is 


Is  always 
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=  0.088k^  -  0.176k^  +  0.088k® 
2  2 

»  0.088k‘^(l  -  kr  >  0  (1-85) 


The  non-Uoear  zero  aejaoiy  filter,  therefore,  always  gives  a  lower 
mean  square  error  than  that  of  the  linear  filter.  As  a  mmerical  ex- 
anq^e,  let  k  »  0.5  (Q  “  0.69^  see).  Vfe  find 


e  0.819; 


(,<'">AV 


0.781 


(1.86) 


*■  Linear 

or,  approximately  a  5?(  iarproveaient. 


J.  5br  purposes  of  illustrating  ths  procedures  outlined  in 
section  k,  we  consider  here  another  example  of  pure  prediction.  Let 
the  desired  output  be  m(tK3:}.  Ihe  aaq>litude  of  the  input  process  at 
any  time  can  take  on  any  one  of  the  four  values  il  and  12  with 
equal  probability.  The  second  order  probability  deaisity  function  is 

JKxi.Xg)  =  i^Cl  ..  e‘Pl‘^l-'2^M{2,-2)-Kl>-lW-2,2W~l,l)] 

+  ^{1  -  e'^^‘'l~''2b[(2,"l)+(l,-2W”2,l)+<“l,a)) 

+  ^(1  -  e"^5^^1”'^2^)[(2,l)+(l,2)+{-2,-l)+(-l,-2)] 

4,  ^(1  4  e"^.'‘^l"'2U  e:'^i^l-^2l4  e"^^‘^l"''2*) 

X  [(2,2)+fl,lH{-2,-2>f(-l,-l}]  (1.87) 

Bere,  we  let  6^^  «  Pg  »  1,  Pj  »  2  and  we  denote  e"® 
optimizing  Integral  equations  axe 


by  k.  The 
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7ke-"  k^e-^ 


The  Fhurler  transforms 


o 

O 

of  the  respective  kernels  are 


0<t<T  (1«88} 


and 


+  k 

x^  +  5X^  -i-  4 


It  follows  that 


r 

10  2 

10 

-2 

[D]  sa 

1 

2  10_ 

and  [D]“^  « 

-2 

10_ 

(1.89) 

(1.9^  ' 

(1.91) 


Following  idle  above  mentioned  procedures,  ve  find  that  (1.41)  in  our 

exan^e  is 


’Ske-"  -  kV^^ 

10(~  ^  +  2.8) 
Qv*- 

10 

-2 

Fjft) 

Tke"'*^  +  -k^e"^ 

«r 

a8 

2(~  f  2) 
dt^ 

10(-  ~  +  2.8) 

-2 

L _ 

0  <  t  <  T  (1.98) 
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which  after  aiirg>llfloation  beccnes 


„  -t  ,2  -2t 
5ke  -  k  e 


_  -t  ,2  -2t 
7ke  +  k  e 


1 


j|2 

a  y. 


.26^^68yi  +  10^-4yg 


dS'o 


a  yi  —  jp 

.i0at2  -Vi'S^^  +  esyg 
0  <  t  <  T 


(1.93) 


The  solution  for  y{t)  is  found  to  be 

yj^(t)  «  Sj2*~'^  ■*■  S-3®^  3*“’ 


(1.9»^) 


ygCt)  -  -  %2*"^  ■  I 


Proceeding  to  the  next 


yt) 


step,  we  obtain  K(t)  frco  (l.lt?): 


'  ^  dt2  ^ 


X 


10 

95 

~2 

55 


-2 

10 

95 


yp(t) 

iSi 


0  <  t  <  T 


(1.95) 


which  after  siJBpllficatiati  becomes 


- 1 

w 

K2(t) 

-Su*  “  -  Bia'"  ^ 

, 

••  -- 

where  and  (1.95) 

are  constants 


Row  let  us  add  bj^^SCt)  +..o^8(t-T}  r-  A  bgj^8(t)  +  Cgj^&(t-T)  to 
I^(t)  and  l^(t)  respectively  and  substitute  the  complete  expression 
into  (1.87).  We  obtain,  (Qxm  lettang  T  *•  100  Billiseconds, 


1 
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«  -2.76  X  10“^  (k  -  k®) 

=  -121.31  X  10"^  (k  -  k®) 

-  (1295k  -  293k®)  X  10'^ 
»  (1707k  4-  293k®)  X  10"^ 
«  -37.2l(k  -  k®)  X  10"^ 


It  ia  clear  that  K^(t)  j(  2X^(t)  (excludlijg  the  case  vhere  k  =  l). 
Consequently,  the  optimum  predictor  for  this  problem  is  always  non¬ 
linear.  It  is  interesting  to  point  ov(t  that  in  this  particular  exanrple, 
one  can  also  obtain  the  same  solution  by  solving  Just  a  single  inte¬ 
gral  equation.  By  ad/^^ng  the  two  equations  in  (1.88),  one  can 
iimedlately  obtain  the  relation  that 

Kj,  (t)  +  KgCt)  »  3000k  X  10’5  6(t)  (l.9fi) 

Upon  substituting  (1.90)  iato  (1.38),  the  two  simultaneous  equations 
con  be  reduced  to  a  single  one. 

The  nozioalised  mean  square  error  for  this  non-Mnear  filter  is 

l  <'‘>.7]  -  1  -  +  Si?  +  »'*)  (1.99:1 

’r.i 

Correspondingly,  ve  find  the  nonnallEed  mean  square  error  for  the 
linear  fl.lter  to  be 

-  1  -  ^[5S.lk®  +  3.8k5  +  O.lk^)  (l.lOO) 

^  J linear 

We  observe  that  the  dlfferexice  between  (l.lOO)  and  (1.99)  is  always 
greater  than  or  equal  to  zero;  that  is 
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^  [0.9k®  -  1.8k^  +  0.9Jt^]  “  55  ^  X  (1  -  k^)  >0  (lolO'i 

Consciquently,  we  have  shown  that  Improved  results  can  he  obtained 
hy  using  a  non™ linear  filter  of  the  class  in  place  of  a  linear 

filter^  The  amount  of  iaprovaaent  depends  on  the  problem  at  hand, 
iibr  the  particular  exanqple  we  Lave  chosen  here  as  well  as  the  numeri¬ 
cal  values  we  have  assumed,  the  laqjro'/ement  is  negligible.  A  k  =  0.5, 
the  improvement  is  about  0.155t.  Inasmuch  as  there  sire  five  para¬ 
meters  involved  in  this  example  3^,  3^,  a  and  T),  it  is 

unlikely  that  any  general  statement  can  be  made  with  regard  to  their 
effects  on  the  mean  square  error  without  first  obtaining  an  explicit 
expression  in  tezsis  of  these  parameters. 
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II.  WJLTIPIJB  FREDIgnOBS 


The  fonoulatlon  of  prohlenus  into  a  set  of  similtaneous  Inte¬ 
gral  equations  occurs  in  many  fields;  for  exaniple,  in  the  study 

of  radiation  and  vave  propagation  In-'/olvlng  quite  general  boundary 
12 

conditions.  One  problem  vhich  is  of  pco^leular  interest  to  us  is  the 
synthesis  of  linear  predictors  and  filters  for  aultiple  tiw 
series.  Briefly^  the  ]^blem  is  as  foUovs: 

Given  a  nuniber  of  stationary  time  series, 

fj^(t)  ■  iiij^(t)  +  nj^(t) 

fgCt)  »  a^(t)  +  UgCt) 

fj^(t)  »  inj^(t)  +  ajj.(t)  (2.1) 

we  wish  to  find  a  set  of  linear  filters  lijj('fc)  so  that  the  sum  over 

all  Inputs  J  (j  •  1,2, ...,]c)  of  the  outputs  of  the  operat¬ 

ing  on  the  past  of  ®j(t)  +  OjCt),  respectively,  is  the  best  approxi¬ 
mation  in  the  least  square  sense  to  n^(t^),  (1  »  1,2, 

Specifically,  let  i  ■  1.  Our  aim  is  to  find  h^(t),  (J  ■■  1,2, ...,h) 
so  as  to  minimize 

k  T 

<^[mj^{t+a)  -  f^(t-T)hj^^(T)dT]^^  (2.2) 

A  block  diagram  of  such  a  k-input,  1-output  network  Is 

shown  in  Fig.  3,  where  ]^(t)  denotes  the  best  estimate  of  m^('bfa). 

Wiener^  has  shown  that  the  irinlmim  of  (2.2)  occurs  when  the 
h^^(T}  satisfy  the  following  set  of  Integral  equations: 

k  T 

Xj(TKt)  ■  5  0_fj(T-o)hj^£(a)do 

0  <  T  <  T;  j  »  1,2,... ,k 

(2.3) 


ng.  3  A  K-l  Pble  network 


vbere  Is  the  correlatlos  function  between  in^(t4o) 

f.Ct-T),  and  0/.  (t-fi)  is  the  correlation  function  between 


and  f^(t-<») 


and 

fj(t-T) 


If  ve  suppose  that  the  Fourier  transforms  of  the  respecti-ve  nels 
are  bounded  at  oo  and  each  Fourier  transform  can  be  represented 
by  a  rational  function  in  JX,  namely. 


then  the  technique  deimloped  in  the  previous  section  is  also  sppUcahle^ 
for  obtaining  the  solution  of  (2.3)- 

By  paralleling  the  steps  in  Appendix  A,  it  is  seen  that  the 
solution  of  (2. 3)  satisfies  a  system  of  differential  equations 

rap(|f)]h(t)  »  QC-  XCtta)  0  <  t  <  T  (2,3) 


vhere 

h(t)  =  a  col.«mn  matrix  whose  elements  are  h^(t),  «  1,2, 

X(-:>K:()=  a  column  matrix  whose  elements  are  Xj(t+a) 

Cj  =  1,2,. ..,k),  Jind 

tD?{^)]  ts  a  k  X  k  square  matrix  whose  elements  are  linear 
operators  )  • 

The  essential  difference  in  the  derivation  of  (2*5)  is  that  here 
it  is  unnecessary  to  break  the  off-diagonal  integrals  into  two  separate 
integrals  lAen  each  differentiation  is  performed.  This  is  because 
only  the  kernels  on  the  di^onal  axis  are  even  function  of  their  argu¬ 
ments.  In  other  words 

0^j(t-o)  »  0fj(lT-o\)  (2.6) 

only  for  ^  *  J. 

If  a  unique,  abeolutely  integrable  solution  exists,  that  is.  If  the 
in^polse  response  of  all  the  linear  filters  shown  in  Fig.  3  satisfies 
the  stability  condition 


the  solution  can  he  obtained  by  adding  delta  functions  located  at 
t  <■  0  and  t  •  T  to  the  solutions  of  (2.^)*  Ibe  unspecified 
constants  can  be  detexmined  by  substituting  the  coo^ete  e^qpression 
for  h(t)  into  (2.3)  end  solving  it  as  an  identity. 
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A-1 


APEEMKDC  A 


We  vleh  to  show  that  the  solution  of 
N  T 

2;^(b+o)  =  E  ^  w^j(}t-T|)K  (T)dT  0<t<T 
J=1  o  j  I  « 


vith  the  condition  that 


.00 


"00 

satisfies  the  foUoving  system  of  differential  equations. 


i  ^  I? 


dt" 

0  <  t  <  T 


A-l 


A.2 


Ao3 


On  accoxuat  of  our  assxauptloa  that  0  is  of  the  form 

h 


A.  4 


it  is  readily  verified,  hy  taking  the  limits  of  the  derivatives  of  any 
order  froen  both  sides  of  t  =  0,  that 


.  (-1)  "  »„(«{o-) 


A.5 


Equation  (A.l)  can  he  written  as 

t 


(t-4a)  B  E 


a 

E 

.3*0. 


0  <  t  <  T;  i  ~ 


Taking  derivatlvea  on  both  sides  vlth  respect  to  time  yields*- 


f  Q  i 

»  w^- refers  to  the  derlvati'/es  vith  respect  to  the  argument  Xo 


A-2 


.  j  [kjW  t.y(0*)  -  Vy(0-)i| 


H  \  t 


(t-'r)K  CT)dT 


j-'l)  'o 
T 


ij 


0  <  t  <  T  Ao7 

i  »  1,2, ...,H 

On  account  of  (A.5),  tie  first  suMDation  vanisbe's  and  ve  are  left  vlth 


H  I  t 


zJ^^Ctta)  =  v^jV-T)Kj(T)dT  -  wW(T-t)Kj(T)d'rj  A«8 

^  0<t<T;l  =  1,2,. ..,H 

Similarly 

|Kj(t)tw£j^C0’*’)  +  w[j^(0")]  j 

+  (t-t)KjCT)dT.{.  \  w[®\T-t)Kj(T)dt|  A.9 


il)/ 


0  <  t  <  T;  1  =  1,  2,  ...,  H 


vhich  in  view  of  (A. 5}  reduces  to 


z![^^Ct4a)  «  I 


\  wJ^^(t-'r}K^(T)dT+  \  wJ®^(T.t)Kj(T)dT 


f  A.  10 


0<t<T;  ial,2,...,H 


Fallowing  the  same  reasoning,  one  can  show  that 
H  I  t 


z^^^(i>ta)  -  Z  \{  wJj^(t-T)Kj(T)dT 
^  T 

+  (-1)^  wW(T-t)Kj(T)dT 


&  <  t  <  T 
■>  i  =  1,2,...,  H 


A.ll 


A-3 


since  the  even  derivative  of  an  even  function  la  again  an  even  func¬ 
tion,  we  can  write  (A.ll)  as 


1?  T 

4^^(t4a)  B  Z  \  w|^^^(\t-'r\)K,(T)aT  for  k  a  even  A.12 

1  1*1  o  J 

0<t<T;i» 


From  the  inverse  Fourier  transform  relationship  of  ,  we 

obtain 


4^(1  X',)  ^  \  ^  e'’^  d\  for  k  =  even  A.IJ 

^  ioo  Q(\2) 


Consequently  if  we  operate  on  both  sidv^s  of  (A<.6)  by  Q(-  ~2^' 
denominator  Q(X^)  vanishes  and  the  resultant  equation  becomes 


dt2^ 


ZlCt-w)  «=  ^ij  fKjCT)aTX  ^ 


OD 

\ 

-CD 


0  <  t  <  T;  i  «  1,2, . . .  ,H  A,14 


Die  numerator  polynomial  )  can  be  built  up  by  differentiating 

the  integral.  Then  (A.l^i)  can  be  written 


dt2 


1? 


j.jCtK.)  .  ^  £  )  i  iCj(T)a,  xi; 

0  <  t  <T;  1  »  1,2,...,H 


or 

«-  - 1  Vu  <■ 

0  <  t  <  T;  i  *  1,2,...,  H 
Where  use  has  been  made  of  the  relation 
00 


A.  15 


1.  c  „  6(lt-Tl) 

-00 


APPEiroiX  B 


In  tbio  appendix  we  carry  out  the  first  few  integrations  of 
(1,53)  and  show  how  it  leads  to  (I056).  Let  ua  rewrite  (1.53), 


-2  ^  w(lt-T|)K  (r)dT  =  ^  2  (-l)^q3j^!^^^(T4a)w(t-T)dT 

J«1  i  ^  o  k=0 

T  d 

+  ^  •£  (-l)\3/J^^(T-K*)0(T-t)dT 


T  d 

^  £ 
t  feO 

0  <  t  <  T;  i  =>  l,2,o«.,N 


Integrating  the  ri^t  hant',  side  of  (B.l)  once  by  peurts  yields 


%  \  y^(T4a)wCl  t-T|)dT 


2  X  X  w(t-t))  I 

sal  I  O 


^  v^i  ^  ^  Wi 
k=l 


^  (8k-l) 


(T^kx)  X  W^-'-^(t-T)dT 


-J-  £  (-l)Vv  X  [y5^'^-^{T4a}  X  w(T-t)) 

kal  ^ 


t  ceiicelo  ranalnlng  tezoft  wmlnatsd 

Xw(t) 

[y(2k-l)^^^j  X  'wC'T-t) 

il  9  Sa^' 

Perfomrijig  the  same  integration  by  parts  on  the  remaining  iatograls 
gives 


the  terms  evaliiated  at  t  =« 
at  t  ■  0  end  t  •  T  are 

-  2  (-I)V  X 

kcl 


+  S  (-1)  <lov 

texl  “ 


£  (-1)^  q..  X  [y5^'^^(T4a)  xO^^'^Ct-T)]  f 

k«l  ^  io 

+  2  (-1)’^  iu  X  \  Xv^'‘^ht‘-x)Ax 

k^i  o  ^ 

-  i  (-1)^  q...  X  [y[2J^'2)  X^Cl)(T-t)]  T 

k-1  ‘■■’^  ^  it 

+  i  (-1)^  qgv.  X  f  yJ^'^^(T4a)  Xv^^\%~t)Ax 
k»l  t  ^ 

0  <  t  <  T;  is  1,2,»..,S  B.5 

The  terns  evaluated  at  ret  again  cancel.  The  two  other  terms  evaluated 
St  t  «  0  and  t  ••  T  are 

-  ^  (-1)^  Vov  X 

k»l  ^  "  \ 

-  s  Ogi- X  [yi^"®^(3Ha)]w^^^(T"t)  iKi,2,,..,i»  b.6 

ksl  ’  “ 


Tae  i-eaairslng  integrals  are  again  IntegrattJd  by  pturta  and  t)ae  process 
is  continued  2d  times,  leaving  integrals 

("1}^  I  yi(T4a)0^®^^(|t-T!)dT  ^  i  =  1,2,.  .,,®  B.? 

unlategrated  as  they  occur  at  every  other  step-  In  addition,  ve  have 
terms  In  th.e  foim  of  (3.4)  and  (B.6)  lef  t  over  et  each  of  the  2d 
integrations.  Inspection  of  (B.4)  and  CBo6)  shows  that  ths  ‘fce:nas  left 
over  can  'be  put  in  a  ecopact  forp...  Let  us  define 
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![£=>■*“  for  X  »  odd 
n  ^  for  J?  K  even 


i  a  1,2,...,® 
B.8 
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2 

Is=!  for  /  a  odd 
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ror 


Jl 


(rfa)  I 

i  .=  1,?, 


BcQ 


ewn 


Qlien'  the  trteJ.  aumbei-  of  tei®n  which  sure  evaluated  at  t  *  0  and  t  =  T 

can  he  written  as 


_ t 


hi 


A^£. 
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i  ^  -^,2, . .  -  ,Tf 


B.IO 


It  is  noted  the,t.  eind 

vatives  of  y^^hStt)  at  t  •-  0 


are  limvir  ccenbicetions  of  the  devr" 
end  t  --  V;  By  tiBlcLn^:  use  t  f  (B.'f  ) 


and  (B.IO),  the  rl^t  hand  side  of  (B.l)  becomes 
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^=1 
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X  S 
kaO 


(-D^qgjV^^^at-xDdT 


vfalch  can  be  Identified  as  the  n^t  band  side  of  (1.^6). 


AHSlffilX  C 


We  wisn  to  shCK-r  that  if  is  the  solution  of  (l-?(5),  it  is 

also  a  solution  of  (1.42)  \6iere  y(t+a)  is  any  solution  satisfying 
the  set  of  differential  equations  (l.4l).  Let  the  solution  of  (1.36) 
he  K(t).  Then 

[DP(-^2)][f  C<lt-T|)K(T)dT] 

2Ct4a)  =■  IDP(-  ^^)][D]"Vj_Ct-Ki!)  C.l 

0  <  t  <  T 

where  yj^(t^-a)  could  be  any  solutton  of  (1.41).  Let  us  introduce 
G(t)  where 

G(t)  =  [f  Oc\t-T!)K(T)d^]  -  [T>r\(ua) 

0 

0  <  t  <  T  Co2 


It  is  sreadily  seen  frcsn  (C.l)  that 


C.3 


It  we  .let 


ziiM  =  Zil±!Sl 


C.4 


-v 

then  we  see;  that  yit-xa)  is  a  solution  of  (1.^14)  if  we  ojcra-te  on 
both  sides  of  (0.4)  by  [DP(-  o- 

CC.l)  shows  that  K(tj  is  the  solution  of  both  equations  when 
■a  which  eatabliahes  the  desired  result  o 
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